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EXECUTIVE  SUMMARY 


Issue  and  Overview 

The  purpose  of  this  report  is  to  provide  evidence-based  recommendations  to  facilitate  the  assignment  of 
personnel  to  foreign  languages  for  initial  acquisition  training  (1AT)  at  the  U.S.  Army  John  F.  Kennedy 
Special  Warfare  Center  and  School  (USAJFKSWCS).  This  inquiry  is  directed  at  determining  the 
strongest  predictor  or  combination  of  predictors  of  foreign  language  learning  as  assessed  on  the  Oral 
Proficiency  Interview  (OP1),  which  is  used  as  the  graduation  standard  at  USAJFKSWCS.  It  also 
recommends  cut  scores  for  various  predictor  options.  Although  cut  scores  can  be  used,  a  top-down 
assignment  approach  using  the  best  predictor  is  often  the  simplest  option  to  execute. 

Findings 

Which  test  (or  combination  of  tests)  is  most  predictive  of  exceeding  the  OPI  standard? 

•  A  weighted  composite  of  the  Defense  Language  Aptitude  Battery  (DLAB)  and  the  Wonderlic 
Personnel  Test  (WPT™)  is  the  best  overall  predictor  of  exceeding 1  the  graduation  standard  on  the 
OPI  (r  =  .18). 

•  DLAB  is  the  best  single  predictor  of  exceeding  standard  on  the  OPI  (r  =  .  17),  followed  by  the 
WPT™  (r  =  .  14)  and  the  Army  General  Technical1 2  (GT)  score  on  the  Armed  Services  Vocational 
Aptitude  Battery  (ASVAB;  r  =  .  13). 

•  The  DLAB-WPT™  composite  does  a  comparable  job  of  identifying  those  individuals  who 
exceeded  the  standard  as  the  DLAB,  but  the  composite  does  a  slightly  better  job  identifying  those 
who  did  not  exceed  the  standard  than  the  DLAB.  This  makes  the  composite  a  slightly  more 
effective  predictor. 

Conclusions  and  Recommendations 

•  In  an  absolute  sense,  none  of  the  individual  tests  or  test  combinations  are  very  good  predictors  of 
exceeding  the  OPI  standard.  These  tests  were  more  predictive  of  Defense  Language  Proficiency 
Test  (DLPT)3  ratings  than  of  OPI  ratings. 

•  Since  the  DLAB-WPT™  composite  1)  only  provides  a  slight  increase  in  prediction  over  and 
above  the  DLAB,  and  2)  the  feasibility  of  using  two  tests  may  not  be  practical: 

o  Recommendation:  Use  the  DLAB  to  set  cut  scores  and  make  decisions  for  assigning  the 
highest  potential  students  to  specific  languages, 
o  Recommendation:  If  DLAB  scores  are  unavailable,  use  the  WPT™  to  make  assignments. 
If  WPT™  scores  are  also  unavailable,  use  the  GT  score. 


1  Due  to  the  high  rate  of  individuals  meeting  the  graduation  standard  on  the  OPI  (i.e.,  little  variability  in  scores), 
exceeding  standard  was  used  as  the  criterion.  Individuals  with  scores  of  1/1+  or  1+/1  were  included  in  the  exceeded 
standard  group. 

2  Given  the  substantial  overlap  between  the  GT  and  the  Aimed  Forces  Qualification  Test  (AFQT;  r  =  .87),  only  the 
GT  was  examined.  Previous  technical  reports  suggested  the  GT  was  better  than  the  AFQT  at  differentiating 
low/high  OPI  performers  at  USAJFKSWCS. 

3  For  details  regarding  predictors  of  the  DLPT,  please  see  SWA  Consulting  Inc.  (2008,  Nov.).  SOF  language 
training  analysis  support  project:  Wonderlic  Personnel  Test  as  a  predictor  of  language  proficiency  (Technical 
Report  #2008010608).  Raleigh,  NC:  SWA  Consulting  Inc. 
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•  Recommendation:  Regardless  of  the  composite  or  individual  test  used,  executing  a  cut  score 
strategy  (i.e.,  only  assigning  personnel  who  exceed  the  minimum  cut  score  to  a  language)  may  not 
be  feasible.  In  cases  when  using  cut  scores  are  not  feasible,  we  recommend  a  top-down 
assignment  protocol  (i.e.,  rank  ordering  trainees  from  highest  to  lowest  on  the  predictor  and 
assigning  the  trainees  with  the  highest  scores  to  the  most  difficult  languages)  because  it  optimizes 
the  assignment  process  and  is  often  the  simplest  method  to  execute. 

If  cut  scores  are  utilized  with  single  test  scores  (see  Table  2,  p.  6  for  DLAB-WPT™  composite  cut 
scores),  the  DLAB,  WPT™,  and  GT  scores  in  the  table  below  are  the  recommended  minimum  scores  for 
assigning  trainees  to  various  target  languages. 


Cut  Scores 

Language 

DLAB 

WPT™ 

GT 

CAT  I 

French 

77 

51 

115 

Spanish 

87 

68 

121 

CAT  II 

Indonesian 

83 

59 

119 

CAT  III 

Dari 

104 

87 

126 

Pa  shtu-  Afghan 

104 

82 

126 

Persian-Farsi 

101 

66 

121 

Russian 

99 

69 

122 

Tagalog  (Filipino) 

67 

15 

104 

Thai 

90 

52 

115 

Urdu 

83 

47 

114 

CAT  IV 

Chinese-Mandarin 

116 

87 

131 

Korean 

98 

52 

116 

Modern  Standard  Arabic 

111 

79 

127 
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REPORT  DETAILS 

This  report  was  produced  as  part  of  the  Special  Operations  Forces  Language  Office  (SOFLO, 
USSOCOM)  training  analysis  support  initiative.  The  purpose  of  this  report  is  to  provide  evidence-based 
recommendations  to  facilitate  the  assignment  of  personnel  to  foreign  languages  for  initial  acquisition 
training  (1AT)  at  the  U.S.  Army  John  F.  Kennedy  Special  Warfare  Center  and  School  (USAJFKSWCS). 

This  study  evaluated  the  Army  General  Technical  (GT)  score  on  the  Armed  Services  Vocational  Aptitude 
Battery  (ASVAB)4,  Defense  Language  Aptitude  Battery  (DLAB),  and  the  Wonderlic  Personnel  Test 
(WPT™)  to  determine  the  strongest  predictor  or  combination  of  predictors  of  foreign  language  learning. 
Specifically,  we  evaluated  how  effectively  these  tests  predict  whether  or  not  students  will  exceed  the 
foreign  language  Oral  Proficiency  Interview  (OP1)  graduation  standard  (i.e.,  achieving  1/1 +,  1+/1,  or 
higher5)  at  USAJFKSWCS.  We  evaluated  the  tests  both  individually  and  as  optimally  weighted 
combinations.  This  report  also  recommends  cut  scores  based  on  these  various  predictor  options. 

Sample 

Participants  in  this  study  were  selected  for  inclusion  based  on  consistently  available  scores  for  each 
predictor  and  OP1  ratings.  The  resulting  sample  included  1,813  students6  from  the  USAJFKSWCS.  The 
sample  contained  data  from  Soldiers  who  studied  various  languages,  ranging  from  Category  (CAT)  1  to 
CAT  IV.  Of  the  1,813  students,  791  exceeded  the  OP1  graduation  standard  and  1,022  did  not  exceed  the 
standard.  However,  of  the  1,022  who  did  not  exceed  the  standard,  995  met  the  1/1  OP1  standard. 

Findings 

When  considered  individually,  DLAB  scores  have  the  strongest  relationship  with  exceeding  the  OP1 
standard  (Table  1,  p.  5).  However,  a  DLAB-WPT™  composite  provided  the  strongest  relationship  when 
compared  to  all  individual  and  combinations  of  predictors  (e.g.,  WPT™  and  GT,  GT  and  DLAB).  Table 
1  (p.  5)  illustrates  that  the  DLAB-WPT™  composite  does  a  comparable  job  of  identifying  those  who 
exceeded  1/1  as  the  DLAB,  and  the  composite  does  a  slightly  better  job  of  identifying  those  who  did  not 
exceed  1/1  than  the  DLAB. 

Overall,  all  tests  were  less  predictive  of  OPI  ratings  than  Defense  Language  Proficiency  Test  (DLPT) 
ratings.7  Additionally,  previous  findings  indicated  an  Armed  Forces  Qualification  Test  (AFQT)  and 
Wpt™  composite  was  the  best  predictor  of  the  DLPT,  whereas  a  DLAB-WPT™  composite  was  the  best 
predictor  of  exceeding  standard  on  the  OPI  {Wonderlic  Personnel  Test  as  a  Predictor  of  Language 


4  Given  the  substantial  overlap  between  the  GT  and  the  Aimed  Force  Qualification  Test  (AFQT;  r  =  .87),  only  the 
GT  was  examined.  Previous  technical  reports  suggested  the  GT  was  better  than  the  AFQT  at  differentiating 
low/high  OPI  performers  at  USAJFKSWCS. 

5  Due  to  the  high  rate  of  individuals  meeting  standard,  exceeding  standard  was  used  as  the  criterion.  Individuals 
with  scores  of  1/1+  or  1+/1  were  included  in  the  exceeded  standard  group. 

6  To  enhance  confidence  in  these  findings,  optimal  cut  scores  were  estimated  using  a  randomly  selected  subsample 
(60%  of  the  hill  sample).  The  resulting  cut  scores  were  evaluated  on  the  remaining  40%  of  the  sample. 

7  For  details  regarding  predictors  of  the  DLPT,  please  see  SWA  Consulting  Inc.  (2008,  Nov.).  SOP  language 
training  analysis  support  project:  Wonderlic  Personnel  Test  as  a  predictor  of  language  proficiency  (Technical 
Report  #2008010608).  Raleigh,  NC:  SWA  Consulting  Inc. 
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Proficiency,  Technical  Report  #2008010608).  However,  for  both  the  DLPT  and  OP1,  DLAB  scores 
provided  the  strongest  relationship  with  meeting  or  exceeding  standard  than  other  individual  tests. 

Table  1.  Relationships  between  Predictors  and  Exceeding  OP1  Standard 


Predictor 

Correlation  with 
Exceeding  l/la 

Percentage  of  high 
scorers  that 
exceeded  l/lb 

Percentage  of  low 
scorers  that  did  not 
exceed  l/lb 

Best  Predictor 

DLAB-WPT™  Composite8 

.18 

66% 

68% 

Other  Predictors 

DLAB 

.17 

67% 

61% 

WPT™ 

.14 

58% 

56% 

GT9 

.13 

57% 

61% 

Based  on  1,813  students. 

“All  correlations  are  statistically  significant  (p  <  .001). 
b  These  comparisons  are  between  students  testing  in  the  same  target  language. 


Table  2  (p.  6)  shows  the  recommended  DLAB-WPT™  composite  cut  scores  for  assigning  the  highest 
potential  students  to  specific  languages  (see  Appendix  A  for  an  example  on  how  to  use  Table  2).  These 
cut  scores  were  selected  to  balance  a)  the  number  of  Soldiers  selected,  and  b)  the  probability  of  success 
(i.e.,  exceeding  standard)  of  those  selected.  Since  the  combination  of  DLAB  and  WPT™  was  the  best 
predictor  of  the  OP1  outcome,  we  present  cut  scores  for  this  single  composite  score  rather  than  for  DLAB 
and  WPT™  individually.  Many  different  combinations  of  DLAB  and  WPT™  scores  will  produce  the 
same  composite  score,  thus  setting  specific  cut  scores  for  either  test  individually  is  not  appropriate  here. 

Table  2  (p.  6)  also  shows  how  these  cut  scores  performed  in  the  40%  subsample10  (i.e.,  what  percentage 
of  the  selected  students  would  have  exceeded  the  OP1  graduation  standard  if  a  cut  score  was  applied). 
Using  trainees’  DLAB-WPT™  composite  scores,  the  cut  scores  specified  for  each  language  should  guide 
assignment  of  trainees  to  the  appropriate  language.  For  instance,  of  the  131  French  students  in  the  40% 
subsample,  38  (29%)  exceeded  the  OP1  graduation  standard.  If  a  cut  score  of  50  was  applied  to  select 
these  students  into  French,  only  60  of  the  131  would  have  met  the  cut  score;  however,  26  of  those  60 
selected  (43%)  would  have  exceeded  the  OP1  graduation  standard.  Thus,  using  the  cut  score  would 
increase  the  rate  at  which  trainees  exceeded  the  OP1  standard  (i.e.,  increasing  the  efficiency  of  training). 

In  general,  although  fewer  students  would  have  been  assigned  to  each  language  had  the  cut  score  been 
used,  the  trainees  who  would  have  been  assigned  to  their  actual  training  language  would  have  exceeded 
the  OP1  standard  at  a  higher  rate.  Persian-Farsi,  Thai,  and  Korean  were  the  only  exceptions.  The  results 
show  that  using  the  cut  score  would  have  produced  poorer  testing  outcomes  for  these  languages,  but  this 


s  This  composite  can  be  calculated  as  [(DLAB  score  +  WPT™  score*. 40)\2]. 

9  Given  the  substantial  overlap  between  the  GT  and  the  Aimed  Force  Qualification  Test  (AFQT;  r  =  .87),  only  the 
GT  was  examined.  Previous  technical  reports  suggested  the  GT  was  better  than  the  AFQT  at  differentiating 
low/high  OPI  performers  at  USAJFKSWCS. 

10  To  enhance  confidence  in  these  findings,  optimal  cut  scores  were  estimated  using  a  randomly  selected  subsample 
(60%  of  the  hill  sample).  The  resulting  cut  scores  were  evaluated  on  the  remaining  40%  of  the  sample. 
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is  not  unexpected.  Even  using  highly  valid  predictors,  suboptimal  decisions  will  be  made  in  some  cases. 
However,  using  validated  cut  scores  will  produce  more  optimal  (or  correct)  decisions  than  suboptimal 
decisions  over  a  large  number  of  trainees  for  all  languages. 

Furthermore,  although  fewer  high  proficiency  Soldiers  were  produced  overall  when  the  cut  scores  were 
applied,  the  overall  process  was  more  efficient  because  a  higher  percentage  of  the  selected  trainees 
exceeded  the  OP1  standard. 

Table  2.  Recommended  DLAB-WPT™  Composite  Cut  Scores  by  Language 


No  Cut  Used 

If  Cut  Used 

Language 

Cut 

Score 

Selected 

Exceeded 

Exceeded 

Selected 

Exceeded 

Exceeded 

CATI 

French 

50 

131 

38 

29% 

60 

26 

43% 

Spanish 

55 

138 

69 

50% 

44 

32 

73% 

CAT  II 

Indonesian 

52 

51 

35 

69% 

27 

21 

78% 

CAT  III 

Dari 

67 

23 

7 

30% 

4 

2 

50% 

Pa  shtu-  Afghan 

67 

50 

41 

82% 

16 

14 

88% 

Persian-Farsi 

65 

15 

3 

20% 

5 

0 

0% 

Russian 

64 

112 

28 

25% 

45 

15 

33% 

Tagalog  (Filipino) 

41 

25 

20 

80% 

22 

18 

82% 

Thai 

56 

10 

8 

80% 

7 

5 

71% 

Urdu 

54 

22 

4 

18% 

15 

3 

20% 

CAT  IV 

Chinese-Mandarin 

74 

39 

25 

64% 

18 

14 

78% 

Korean 

62 

22 

3 

14% 

17 

2 

12% 

Modern  Standard  Arabic 

71 

115 

62 

54% 

52 

30 

58% 

These  results  are  based  on  the  40%  subsample,  753  students. 


Since  the  DLAB-WPT™  composite  only  provides  only  a  slight  increase  in  prediction  of  the  OP1  outcome 
over  and  above  the  DLAB  alone  and  the  feasibility  of  using  two  tests  may  not  be  practical,  the  remainder 
of  this  report  provides  recommended  cut  scores  for  each  individual  predictor.  The  individual  predictor 
cut  scores  can  also  be  used  in  place  of  any  missing  predictor.  For  instance,  although  the  DLAB  was  the 
single  best  predictor,  if  a  trainee’s  DLAB  score  is  unavailable,  using  the  WPT™  score  would  be  the  next 
best  option,  followed  by  the  GT  score. 

Table  3  (p.  7)  provides  recommended  cut  scores  for  assigning  trainees  to  each  language  when  only  the 
DLAB  is  used  as  the  predictor  (see  Appendix  A  for  a  detailed  example  of  how  to  use  Table  3).  Overall 
performance  of  the  DLAB  cut  scores  is  comparable  to  that  of  the  DLAB-WPT™  cut  scores.  Consistent 
with  the  composite  cut  scores,  those  who  met  the  cut  score  for  their  training  language  exceeded  the  OP1 
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graduation  standard  at  a  higher  rate  than  those  who  did  not  meet  the  cut  score.  Pashtu-Afghan,  Persian- 
Far  si,  and  Korean  were  the  only  exceptions. 

Table  3.  Recommended  DLAB  Cut  Scores  by  Language 


No  Cut  Used 

If  Cut  Used 

Language 

Cut 

Score 

Selected 

Exceeded 

Exceeded 

Selected 

Exceeded 

Exceeded 

CAT  I 

French 

77 

131 

38 

29% 

58 

24 

41% 

Spanish 

87 

138 

69 

50% 

41 

29 

71% 

CAT  II 

Indonesian 

83 

51 

35 

69% 

26 

20 

77% 

CAT  III 

Dari 

104 

23 

7 

30% 

7 

4 

57% 

Pashtu-Afghan 

104 

50 

41 

82% 

17 

13 

77% 

Persian-Farsi 

101 

15 

3 

20% 

5 

0 

0% 

Russian 

99 

112 

28 

25% 

40 

15 

38% 

Tagalog  (Filipino) 

67 

25 

20 

80% 

21 

17 

81% 

Thai 

90 

10 

8 

80% 

7 

6 

86% 

Urdu 

83 

22 

4 

18% 

15 

3 

20% 

CAT  IV 

Chinese-Mandarin 

116 

39 

25 

64% 

20 

15 

75% 

Korean 

98 

22 

3 

14% 

17 

2 

12% 

Modern  Standard  Arabic 

111 

115 

62 

54% 

39 

23 

59% 

These  results  are  based  on  the  40%  subsample,  753  students. 


In  comparison,  the  recommended  DLAB  cut  scores  based  on  the  data  in  this  study  are  fairly  consistent 
with  the  minimum  recommended  DLAB  cut  scores  for  each  language  category  as  set  by  the  Department 
of  Defense  (DoD;  R1 1  -6  Army  Linguist  Management;  Table  4,  p.  7).  However,  both  the  recommended 
scores  in  this  study  and  those  suggested  by  the  DoD  are  higher  than  those  provided  in  the  USSOCOM 
M350-8  (2009). 

Table  4.  DoD  and  USSOCOM  Recommended  DLAB  Scores 


Language  Category 

DLAB  Score  from  DoD 

DLAB  Score  from 
USSOCOM 

CAT  I 

85 

75 

CAT  II 

90 

75 

CAT  III 

95 

80 

CAT  IV 

100 

85 

If  DLAB  scores  are  unavailable  for  trainees,  WPT™  scores  are  the  next  best  option  for  assigning 
personnel  to  a  training  language.  Table  5  (p.  8)  provides  the  recommended  WPT™  cut  scores  for 
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assigning  personnel  into  various  languages  (see  Appendix  A  for  a  detailed  example  on  how  to  use  Table 
4).  Consistently,  those  who  met  the  cut  score  for  their  assigned  language  tended  to  exceed  the  OP1 
graduation  standard  at  a  higher  rate  than  those  who  did  not  meet  the  cut  score.  Persian-Farsi  and  Thai 
were  the  only  exceptions. 

Table  5.  Recommended  WPT™  Cut  Scores  by  Language 


No  Cut  Used 

If  Cut  Used 

Language 

Cut 

Score 

Selected 

Exceeded 

Exceeded 

Selected 

Exceeded 

Exceeded 

CATI 

French 

51 

131 

38 

29% 

69 

25 

36% 

Spanish 

68 

138 

69 

50% 

41 

28 

68% 

CAT  II 

Indonesian 

59 

51 

35 

69% 

23 

17 

74% 

CAT  III 

Dari 

87 

23 

7 

30% 

6 

4 

67% 

Pashtu-Afghan 

82 

50 

41 

82% 

15 

13 

87% 

Persian-Farsi 

66 

15 

3 

20% 

9 

1 

11% 

Russian 

69 

112 

28 

25% 

65 

19 

29% 

Tagalog  (Filipino) 

15 

25 

20 

80% 

25 

20 

80% 

Thai 

52 

10 

8 

80% 

6 

4 

67% 

Urdu 

47 

22 

4 

18% 

15 

3 

20% 

CAT  IV 

Chinese-Mandarin 

87 

39 

25 

64% 

11 

9 

82% 

Korean 

52 

22 

3 

14% 

17 

3 

18% 

Modern  Standard  Arabic 

79 

115 

62 

54% 

59 

34 

58% 

These  results  are  based  on  the  40%  subsample,  753  students. 


If  DLAB  and  WPT™  scores  are  unavailable  for  trainees,  GT  scores  are  the  next  best  option  for  assigning 
personnel  to  a  training  language.  Table  6  (p.  9)  provides  the  recommended  GT  cut  scores  for  assigning 
personnel  into  various  languages  (see  Appendix  A  for  a  detailed  example  on  how  to  use  Table  5).  As 
shown  with  other  predictors,  those  who  met  the  cut  score  for  their  assigned  language  exceeded  the  OP1 
graduation  standard  at  a  higher  rate  than  those  who  did  not  meet  the  cut  score.  However,  there  were  more 
exceptions  to  exceeding  at  a  higher  rate  than  when  the  other  cut  scores  were  applied  (e.g.,  using  DLAB 
cut  scores).  Persian-Farsi,  Tagalog,  Urdu,  Chinese-Mandarin,  and  Korean  were  all  exceptions  to  this 
increase  when  GT  cut  scores  were  applied  for  language  assignment. 
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Table  6.  Recommended  GT  Cut  Scores  by  Language 


No  Cut  Used 

If  Cut  Used 

Language 

Cut 

Score 

Selected 

Exceeded 

Exceeded 

Selected 

Exceeded 

Exceeded 

CAT  I 

French 

115 

131 

38 

29% 

64 

21 

33% 

Spanish 

121 

138 

69 

50% 

38 

25 

66% 

CAT  II 

Indonesian 

119 

51 

35 

69% 

21 

15 

71% 

CAT  III 

Dari 

126 

23 

7 

30% 

8 

4 

50% 

Pa  shtu- Afghan 

126 

50 

41 

82% 

15 

13 

87% 

Persian-Farsi 

121 

15 

3 

20% 

9 

1 

11% 

Russian 

122 

112 

28 

25% 

46 

16 

35% 

Tagalog  (Filipino) 

104 

25 

20 

80% 

24 

19 

79% 

Thai 

115 

10 

8 

80% 

6 

5 

83% 

Urdu 

114 

22 

4 

18% 

17 

2 

12% 

CAT  IV 

Chinese-Mandarin 

131 

39 

25 

64% 

14 

7 

50% 

Korean 

116 

22 

3 

14% 

16 

2 

13% 

Modern  Standard  Arabic 

127 

115 

62 

54% 

42 

24 

57% 

These  results  are  based  on  the  40%  subsample,  753  students. 


Conclusions  and  Recommendations 

In  an  absolute  sense,  none  of  the  individual  predictors  or  composites  were  very  good  predictors  of 
exceeding  the  OP1  graduation  standard  (i.e.,  there  were  small,  though  significant  relationships  between 
the  predictors  and  exceeding  the  OP1  standard).  DLAB  had  the  single  strongest  observed  relationship 
with  exceeding  standard,  but  the  weighted  DLAB-WPT™  composite  improves  slightly  upon  the  DLAB 
on  its  own.  However,  since  there  is  only  a  slight  increase  in  prediction  and  the  feasibility  of  using  the 
combination  of  two  selection  tests  may  not  be  practical,  we  recommend  using  the  DLAB  cut  scores  to 
assign  trainees  to  specific  languages. 

If  DLAB  scores  are  unavailable  for  incoming  language  students,  WPT™  cut  scores  would  be  the  next 
best  option  to  make  language  assignment  decisions.  Furthermore,  if  DLAB  and  WPT™  scores  are 
unavailable,  use  the  GT  score.  Regardless  of  the  composite  or  individual  test  used,  executing  a  cut  score 
strategy  (i.e.,  only  assigning  personnel  who  exceed  the  minimum  cut  score  to  a  language)  may  not  be 
feasible,  especially  if  the  cut  scores  are  different  for  each  language.  In  cases  when  using  cut  scores  are 
not  feasible,  we  recommend  a  top-down  assignment  protocol  (i.e.,  rank  ordering  trainees  from  highest  to 
lowest  on  the  predictor  and  assigning  the  trainees  with  the  highest  scores  to  the  most  difficult  languages) 
because  it  optimizes  the  assignment  process  and  is  straightforward  to  execute. 
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Using  cut  scores  to  guide  language  assignment  is  about  achieving  greater  efficiency  in  the  training 
process.  Language  assignment  using  valid  predictors  such  as  those  identified  in  this  report  helps  achieve 
efficiency  by  identifying  high  potential  learners  early  in  the  process  (i.e.,  predicting  those  who  will 
exceed  the  OP1  graduation  standard  for  a  given  language).  Evidence  suggests  that  using  the 
recommended  DLAB  cut  scores  to  guide  language  assignment  will  improve  efficiency,  in  that  a  higher 
percentage  of  students  will  exceed  the  OP1  standard  and  fewer  will  ‘fail’  (i.e.,  not  exceed  the  standard). 
Furthermore,  these  cut  scores  are  fairly  consistent  with  the  DoD  minimum  DLAB  requirements  across 
language  categories. 

Balancing  the  current  findings  with  the  ease  of  execution,  the  most  straightforward  recommendation  is  to 
use  a  single  test  score  (DLAB  or  WPT™  would  be  best  choices)  and  a  top-down  assignment  protocol 
within  each  military  occupational  specialty  (MOS).  That  is,  ranking  trainees  based  on  their  test  score 
from  highest  to  lowest  within  each  MOS  and  assigning  the  highest  scoring  trainees  in  each  MOS  to 
languages  in  order  of  difficulty  (i.e.,  start  filling  most  difficult  languages  with  trainees  with  the  highest 
scores  in  each  MOS  group  until  MOS  requirements  are  filled  for  each  language,  then  move  to  the  next 
language).  This  is  the  simplest  way  to  optimize  the  assignment  of  your  personnel  to  training  and  optimize 
the  resulting  capability,  especially  when  you  are  using  the  scores  for  assignment  (not  selection). 
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ABOUT  SWA  CONSULTING  INC. 

SWA  Consulting  Inc.  (formerly  Surface,  Ward,  and  Associates)  provides  analytics  and  evidence-based 
solutions  for  clients  using  the  principles  and  methods  of  industrial/organizational  (I/O)  psychology.  Since 
1997,  SWA  has  advised  and  assisted  corporate,  non-profit  and  governmental  clients  on: 

•  Training  and  development 

•  Performance  measurement  and  management 

•  Organizational  effectiveness 

•  Test  development  and  validation 

•  Program/training  evaluation 

•  Work/job  analysis 

•  Needs  assessment 

•  Selection  system  design 

•  Study  and  analysis  related  to  human  capital  issues 

•  Metric  development  and  data  collection 

•  Advanced  data  analysis 


One  specific  practice  area  is  analytics,  research,  and  consulting  on  foreign  language  and  culture  in  work 
contexts.  In  this  area,  SWA  has  conducted  numerous  projects,  including  language  assessment  validation 
and  psychometric  research;  evaluations  of  language  training,  training  tools,  and  job  aids;  language  and 
culture  focused  needs  assessments  and  job  analysis;  and  advanced  analysis  of  language  research  data. 

Based  in  Raleigh,  NC,  and  led  by  Drs.  Eric  A.  Surface  and  Stephen  J.  Ward,  SWA  now  employs  close  to 
twenty  I/O  professionals  at  the  masters  and  PhD  levels.  SWA  professionals  are  committed  to  providing 
clients  the  best  data  and  analysis  upon  which  to  make  evidence-based  decisions.  Taking  a  scientist- 
practitioner  perspective,  SWA  professionals  conduct  model-based,  evidence-driven  research  and 
consulting  to  provide  the  best  answers  and  solutions  to  enhance  our  clients’  mission  and  business 
objectives.  SWA  has  competencies  in  measurement,  data  collection,  analytics,  data  modeling,  systematic 
reviews,  validation,  and  evaluation. 

For  more  information  about  SWA,  our  projects,  and  our  capabilities,  please  visit  our  website  (www.swa- 
consulting.com)  or  contact  Dr.  Eric  A.  Surface  (esurface@swa-consulting.com)  or  Dr.  Stephen  J.  Ward 
(sward@swa-consulting.com). 
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APPENDIX  A:  EXAMPLES 

Appendix  A  provides  several  different  examples  of  trainee  selection  and  language  assignment.  The 
following  example  illustrates  an  example  using  the  DLAB-WPT™  composite  cut  scores  from  Table  2  (p. 
6).  However,  a  similar  process  would  be  used  with  any  composite  or  test  score. 

Appendix  A,  Table  1.  Recommended  DLAB-WPT™  Composite  Cut  Scores  by  Language 


No  Cut  Used 

If  Cut  Used 

Language 

Cut 

Score 

Selected 

Exceeded 

Exceeded 

Selected 

Exceeded 

Exceeded 

CAT  I 

French 

50 

131 

38 

29% 

60 

26 

43% 

Spanish 

55 

138 

69 

50% 

44 

32 

73% 

CAT  II 

Indonesian 

52 

51 

35 

69% 

27 

21 

78% 

CAT  III 

Dari 

67 

23 

7 

30% 

4 

2 

50% 

Pa  shtu- Afghan 

67 

50 

41 

82% 

16 

14 

88% 

Persian-Farsi 

65 

15 

3 

20% 

5 

0 

0% 

Russian 

64 

112 

28 

25% 

45 

15 

33% 

Tagalog  (Filipino) 

41 

25 

20 

80% 

22 

18 

82% 

Thai 

56 

10 

8 

80% 

7 

5 

71% 

Urdu 

54 

22 

4 

18% 

15 

3 

20% 

CAT  IV 

Chinese-Mandarin 

74 

39 

25 

64% 

18 

14 

78% 

Korean 

62 

22 

3 

14% 

17 

2 

12% 

Modern  Standard  Arabic 

71 

115 

62 

54% 

52 

30 

58% 

Using  the  recommended  DLAB-WPT™  composite  cut  scores,  how  would  you  place  the  following 
individuals  into  language  training? 

Scenario 

There  are  4  available  1AT  slots  and  4  available  trainees  with  the  following  sets  of  scores: 

•  Trainee  1 :  DL AB  =105,  WPT™  =  60 

•  Trainee  2:  DLAB  =  75,  WPT™  =  96 

•  Trainee  3 :  DLAB  =  80,  WPT™  =  55 

•  Trainee  4:  DLAB  =  60,  WPT™  =  85 


12/5/2011 


©  SWA  Consulting  Inc.,  2011 
Technical  Report  [2011010631] 


Page  12 


SOFLO  Support  Project 


Evaluating  Predictors  of  Foreign  Language  Learning 


From  a  cut  score  perspective,  the  following  steps  should  be  considered. 

1.  Using  the  composite  equation  [(DLAB  score  +  WPT™  score*.40)\2],  calculate  each  trainees 
composite  score. 

a.  Trainee  1  =  65 

b.  Trainee  2  =  57 

c.  Trainee  3  =  51 

d.  Trainee  4  =  47 

2.  Using  the  DLAB-WPT™  composite  cut  scores  in  the  table  above,  the  4  trainees  should  be 
assigned  into  the  following  languages: 

a.  Trainee  1  -  Persian-Farsi  because  he  meets  the  cut  score. 

b.  Trainee  2  -  Thai  because  he  slightly  exceeds  the  Thai  cut  score  of  56,  but  does  not  reach 
the  next  cut  score  of  62  for  Korean. 

c.  Trainee  3  -  French  because  he  slightly  exceeds  the  French  cut  score  of  50,  but  does  not 
reach  the  next  cut  score  of  52  for  Indonesian. 

d.  Trainee  4  -  Tagalog  because  he  exceeds  the  Tagalog  cut  score  of  41,  but  does  not  reach 
the  next  cut  score  of  50  for  French. 

In  this  case,  if  a  top-down  assignment  approach  had  been  used,  the  same  assignments  would  NOT  have 
resulted  because  of  Tagalog.  Trainee  3  would  have  been  assigned  to  Tagalog  instead  of  French,  and 
Trainee  4  would  have  been  assigned  to  French  instead  of  Tagalog.  Using  cut  scores  does  allow  you  to 
take  advantage  of  situations  in  which  the  traditional  language  difficulty  ratings  do  not  reflect  the  reality  of 
language  learning  at  USAJFKSWCS. 

The  next  examples  focus  on  the  use  of  single  test  scores  (not  a  composite)  but  the  basic  process  is  the 
same. 
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The  next  set  of  examples  apply  to  Tables  3,  5  and  6  (pp.  7-9)  or  single  tests  used  for  assignment; 
however,  the  examples  here  will  focus  on  using  the  recommended  DLAB  cut  scores. 

Appendix  A,  Table  2.  Recommended  DLAB  Cut  Scores  by  Language 


No  Cut  Used 

If  Cut  Used 

Language 

Cut 

Score 

Selected 

Exceeded 

Exceeded 

Selected 

Exceeded 

Exceeded 

CATI 

French 

77 

131 

38 

29% 

58 

24 

41% 

Spanish 

87 

138 

69 

50% 

41 

29 

71% 

CAT  II 

Indonesian 

83 

51 

35 

69% 

26 

20 

77% 

CAT  III 

Dari 

104 

23 

7 

30% 

7 

4 

57% 

Pa  shtu- Afghan 

104 

50 

41 

82% 

17 

13 

77% 

Persian-Farsi 

101 

15 

3 

20% 

5 

0 

0% 

Russian 

99 

112 

28 

25% 

40 

15 

38% 

Tagalog  (Filipino) 

67 

25 

20 

80% 

21 

17 

81% 

Thai 

90 

10 

8 

80% 

7 

6 

86% 

Urdu 

83 

22 

4 

18% 

15 

3 

20% 

CAT  IV 

Chinese-Mandarin 

116 

39 

25 

64% 

20 

15 

75% 

Korean 

98 

22 

3 

14% 

17 

2 

12% 

Modern  Standard  Arabic 

111 

115 

62 

54% 

39 

23 

59% 

Using  the  recommended  DLAB  cut  scores,  how  would  you  place  the  following  individuals  into 
language  training? 

Scenario  1:  Top-down  assignment  approach 

There  are  8  available  1AT  slots  and  8  available  trainees  with  the  following  DLAB  scores:  70,  75,  80,  84, 
95,  102,  117,  and  120.  From  a  top-down  perspective,  the  following  steps  should  be  considered. 

1.  Rank-order  the  trainees  DLAB  scores  from  highest  to  lowest:  120,  1 17,  102,  95,  84,  80,  75,  70 

2.  Assuming  there  is  only  one  slot  available  for  each  language,  trainees  with  the  highest  DLAB 
scores  should  be  placed  into  the  more  difficult  languages. 

3.  The  trainees  with  DLAB  scores  of  120,  1 17,  and  102  should  be  placed  in  the  three  CAT  IV 
languages,  in  order  of  difficulty  with  120  in  the  most  difficult  language. 
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4.  The  5  remaining  trainees  should  all  be  placed  in  CAT  111  languages.  The  trainee  scoring  95 

placed  into  the  most  difficult  CAT  111  language,  trainee  scoring  84  in  the  next  most  difficult  CAT 
III  language,  etc. 

Scenario  2:  Cut  score  approach 

There  are  5  available  1AT  slots  and  6  available  trainees  with  the  following  DLAB  scores:  62,  77,  83,  90, 
103,  andl  12.  From  a  cut  score  perspective,  the  following  steps  should  be  considered. 

1.  The  trainee  with  a  DLAB  score  of  62  can  immediately  be  cut  because  he  falls  below  the  lowest 
cut  score  of  67  (i.e.,  Tagalog). 

2.  The  remaining  5  trainees  should  be  rank  ordered  from  highest  to  lowest:  1 12,  103,  90,  83,  77 

3.  The  trainee  with  a  DLAB  of  1 12  should  be  placed  into  Modem  Standard  Arabic  because  he 
exceeds  the  111  cut  score,  but  does  not  meet  the  116  cut  for  Chinese-Mandarin. 

4.  The  trainee  with  a  DLAB  of  103  should  be  placed  into  Persian-Farsi  because  he  exceeds  the  101 
cut  score,  but  does  not  meet  the  1 04  cut  score  for  Dari  or  Pashtu-Afghan. 

5.  The  trainee  with  a  DLAB  of  90  should  be  placed  into  Thai  because  he  meets  the  cut  score. 

6.  The  trainee  with  a  DLAB  of  83  could  be  placed  into  Indonesian  or  Urdu.  However,  using  the 
top-down  assignment  protocol,  he  should  be  placed  into  Urdu  because  it  is  a  more  difficult 
language. 

7.  Lastly,  the  trainee  with  a  DLAB  of  77  should  be  placed  into  French  because  he  meets  the  cut 
score. 

It  should  be  noted  that  top-down  and  cut  score  approaches  can  produce  slightly  different  results.  In 
general,  the  top-down  approach  is  easier  to  execute,  but  the  cut  score  approach  will  be  more  refined. 
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